Displaying HDMI content at an arbitrary location

ABSTRACT

An audio/video (A/V) hub is described. This A/V hub determines display instructions specifying information to be displayed on a display in an A/V display device that includes icons associated with content sources. These content sources are located at arbitrary or different locations in a structure, including locations external to an environment of the A/V hub and the A/V display device. Then, the A/V hub provides the display instructions to the A/V display device for display on the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. 119(e) to U.S.Provisional Application Ser. No. 62/211,825, “Displaying HDMI Content ina Tiled Window,” filed on Aug. 30, 2015, the contents of which areherein incorporated by reference.

BACKGROUND

Field

The described embodiments relate to display techniques, includingdisplaying high-definition multimedia-interface (HDMI) content from asource at an arbitrary location.

Related Art

The versatility and capabilities of consumer-electronics orentertainment devices is increasing their popularity. For example, thecommunication capabilities of these entertainment devices allow users toaccess content from a wide variety of sources, including high-definitioncontent.

However, while the entertainment devices typically includehigh-resolution displays that allow users to view high-definitioncontent, the interface circuits and the communication bandwidth in manyentertainment devices can pose obstacles to simultaneous viewing ofhigh-definition content.

In addition, the user interfaces associated with many entertainmentdevices can be difficult to use. For example, the process of identifyingcontent from a particular source, selecting the content and having thecontent piped to a particular display often requires that users performmultiple operations. This convoluted process is time-consuming andcumbersome. Moreover, users often make mistakes when attempting tonavigate through a complicated set of options in different menus, whichfrustrates users and degrades their user experience.

SUMMARY

The described embodiments include an audio/video (A/V) hub. This A/V hubincludes: an interface circuit that, during operation, communicates withan A/V display device and one or more content sources; and a controlcircuit coupled to the interface circuit. Note that the content sourcesare located at arbitrary or different locations in a structure,including locations external to an environment of the A/V hub and theA/V display device. During operation the interface circuit determinesdisplay instructions specifying information to be displayed on a displayin the A/V display device that includes icons associated with thecontent sources. Then, the control circuit provides, via the interfacecircuit, the display instructions to the A/V display device for displayon the display.

The A/V hub may include a port that is coupled to the interface circuitand the A/V display device. This port may be compatible with an HDMIstandard.

Moreover, the environment may include one or more rooms in thestructure, and the arbitrary or different locations may be located indifferent rooms in the structure than the A/V hub and the A/V displaydevice.

Furthermore, the A/V hub may include an antenna coupled to the interfacecircuit, and the content sources may be included in a wireless networkassociated with the A/V hub.

Additionally, during operation the interface circuit may communicatewith a portable electronic device, and the control circuit may: receive,via the interface circuit, user-interface activity information from theportable electronic device that specifies user selection of a contentsource. Then, the control circuit may: selectively provide, via theinterface circuit, an activation command to the content source based ondevice-state information of the content source; provide, via theinterface circuit, a request for HDMI content to the content source;receive, via the interface circuit, the HDMI content from the contentsource; and provide, the HDMI content and second display instructions tothe A/V display device as frames with the HDMI content are received fromthe content source, so that the HDMI content is displayed on the displayin the A/V display device. Note that providing the HDMI content and thesecond display instructions may involve transcoding the HDMI contentbased on a format of the display.

For example, the user selection may involve activation of a virtualcommand icon. Alternatively, the user selection may involve activationof a physical button. In some embodiments, during operation the controlcircuit: generates user-interface information that specifies a userinterface that includes one or more virtual command icons, including thevirtual command icon, which are associated with the content sources; andprovides the user-interface information to the portable electronicdevice for display on a touch-sensitive display in the portableelectronic device.

Moreover, the portable electronic devices may include: a remote controlfor the A/V hub; and/or a cellular telephone.

Furthermore, during operation the control circuit may determine thesecond display instructions based on the format of the display.

In some embodiments, the control circuit includes: a processor; and amemory, coupled to the processor, which stores a program module that,during operation, is executed by the processor. The program module mayinclude instructions for at least some operations performed by thecontrol circuit.

Another embodiment provides a computer-program product for use with theA/V hub. This computer-program product includes instructions for atleast some of the operations performed by the A/V hub.

Another embodiment provides a method for providing display instructions.This method includes at least some of the operations performed by theA/V hub.

Another embodiment provides the A/V display device.

Another embodiment provides the portable electronic device.

This Summary is provided merely for purposes of illustrating someexemplary embodiments, so as to provide a basic understanding of someaspects of the subject matter described herein. Accordingly, it will beappreciated that the above-described features are merely examples andshould not be construed to narrow the scope or spirit of the subjectmatter described herein in any way. Other features, aspects, andadvantages of the subject matter described herein will become apparentfrom the following Detailed Description, Figures, and Claims.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram illustrating a system with electronic deviceswirelessly communicating in accordance with an embodiment of the presentdisclosure.

FIG. 2 is a flow diagram illustrating a method for providinghigh-definition multimedia-interface (HDMI) content in accordance withan embodiment of the present disclosure.

FIG. 3 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 4 is a flow diagram illustrating a method for providing HDMIcontent in accordance with an embodiment of the present disclosure.

FIG. 5 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 6 is a drawing illustrating a user interface in a portableelectronic device of FIG. 1 in accordance with an embodiment of thepresent disclosure.

FIG. 7 is a drawing illustrating a display in the audio/video (A/V)display device of FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 8 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 9 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 10 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 11 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 12 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 13 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 14 is a flow diagram illustrating a method for providing displayinstructions in accordance with an embodiment of the present disclosure.

FIG. 15 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 16 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 17 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 18 is a flow diagram illustrating a method for casting of HDMIcontent in accordance with an embodiment of the present disclosure.

FIG. 19 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 20 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 21 is a block diagram illustrating a state-detection circuit in oneof the electronic devices of FIG. 1 in accordance with an embodiment ofthe present disclosure.

FIG. 22 is a flow diagram illustrating a method for detecting anentertainment device in accordance with an embodiment of the presentdisclosure.

FIG. 23 is a block diagram illustrating one of the electronic devices ofFIG. 1 in accordance with an embodiment of the present disclosure.

Note that like reference numerals refer to corresponding partsthroughout the drawings. Moreover, multiple instances of the same partare designated by a common prefix separated from an instance number by adash.

DETAILED DESCRIPTION

In a first group of embodiments, in response to receiving user-interfaceactivity information from a portable electronic device (such as acellular telephone or a remote control) that specifies user selection ofa content source, an audio/video (A/V) hub provides a request forhigh-definition multimedia-interface (HDMI) content to the contentsource based on the user selection. When the A/V hub receives the HDMIcontent from the content source, the A/V hub provide the HDMI contentand display instructions to an A/V display device (such as aconsumer-electronic device, e.g., a television) as frames with the HDMIcontent are received from the content source, so that the HDMI contentis displayed in a tiled window on the display in the A/V display devicewhile other HDMI content from another content source is displayed on thedisplay.

In a second group of embodiments, an A/V hub determines displayinstructions specifying information to be displayed on a display in anA/V display device that includes icons associated with content sources.These content sources are located at arbitrary or different locations ina structure, including locations external to an environment thatincludes the A/V hub and the A/V display device. Then, the A/V hubprovides the display instructions to the A/V display device for displayon the display.

In a third group of embodiments, an A/V hub determines displayinstructions specifying information to be displayed on a display in anA/V display device that includes icons associated with content sources.Then, the A/V hub provides the display instructions to the A/V displaydevice for display on the display. In response to receiving a userselection of a content source based on activation of a single commandfeature in a portable electronic device that is associated with thecontent source, the A/V hub provides a request for HDMI content to thecontent source. When the A/V hub receives the HDMI content from thecontent source, the A/V hub provide the HDMI content and second displayinstructions to an A/V display device as frames with the HDMI contentare received from the content source, so that the HDMI content isdisplayed in a tiled window on the display in the A/V display device.

In a fourth group of embodiments, in response to receivinguser-interface activity information from a portable electronic devicethat specifies user instruction to cast HDMI content associated with acontent source from a first A/V display device to a second A/V displaydevice, an A/V hub accesses device-state information that specifies acurrent state of the second A/V display device. Then, the A/V hubperforms the casting of the HDMI content from the first A/V displaydevice to the second A/V display device using a dynamic number ofoperations based on the device-state information, so that the HDMIcontent is displayed on a display in the second A/V display device.

By dynamically adapting the displayed HDMI content in response to userselections, this display technique may make it easier for a user tocontrol an entertainment device (such as the A/V hub, the A/V displaydevice and/or one or more content sources) with fewer errors ormistakes. Moreover, by providing a simple and intuitive user interfacein the portable electronic device and display instructions for the A/Vdisplay device, this display technique may make it easier for a user to:select HDMI content from different content sources, change how the HDMIcontent is displayed and/or where the HDMI content is displayed.Consequently, the display technique may reduce user frustration and/ormay improve the user experience when using the portable electronicdevice, the A/V hub, the A/V display device and/or one or more contentsources.

In the discussion that follows the portable electronic device, the A/Vhub, the A/V display device, the one or more content sources, and/or anentertainment device (such as a consumer-electronic device) may includeradios that communicate packets or frames in accordance with one or morecommunication protocols, such as: an Institute of Electrical andElectronics Engineers (IEEE) 802.11 standard (which is sometimesreferred to as ‘Wi-Fi®,’ from the Wi-Fi® Alliance of Austin, Tex.),Bluetooth® (from the Bluetooth Special Interest Group of Kirkland,Wash.), a cellular-telephone communication protocol, anear-field-communication standard or specification (from the NFC Forumof Wakefield, Mass.), and/or another type of wireless interface. In thediscussion that follows, Wi-Fi is used as an illustrative example. Forexample, the cellular-telephone communication protocol may include ormay be compatible with: a 2^(nd) generation of mobile telecommunicationtechnology, a 3^(rd) generation of mobile telecommunications technology(such as a communication protocol that complies with the InternationalMobile Telecommunications-2000 specifications by the InternationalTelecommunication Union of Geneva, Switzerland), a 4^(th) generation ofmobile telecommunications technology (such as a communication protocolthat complies with the International Mobile Telecommunications Advancedspecification by the International Telecommunication Union of Geneva,Switzerland), and/or another cellular-telephone communication technique.In some embodiments, the communication protocol includes Long TermEvolution or LTE. However, a wide variety of communication protocols maybe used. In addition, the communication may occur via a wide variety offrequency bands. Note that the portable electronic device, the A/V hub,the A/V display device, the one or more content sources, and/or anentertainment device may communicate using infra-red communication thatis compatible with an infra-red communication standard (includingunidirectional or bidirectional infra-red communication).

Communication among electronic devices is shown in FIG. 1, whichpresents a block diagram illustrating a system 100 with a portableelectronic device 110 (such as a remote control or a cellulartelephone), an A/V hub 112, one or more A/V display devices 114 (such asa television, a monitor, a computer and, more generally, a displayassociated with an electronic device) and one or more content sources116 (e.g., a radio receiver, a video player, a satellite receiver, anaccess point that provides a connection to a wired network such as theInternet, a media or a content source, a consumer-electronic device, anentertainment device, a set-top box, over-the-top content delivered overthe Internet or a network without involvement of a cable, satellite ormultiple-system operator, etc.). (Note that A/V hub 112, the one or moreA/V display devices 114, and the one or more content sources 116 aresometimes collectively referred to as ‘components’ in system 100.However, A/V hub 112, the one or more A/V display devices 114, and theone or more content sources 116 are sometimes referred to asentertainment devices.) In particular, portable electronic device 110and A/V hub 112 may communicate with each other using wirelesscommunication, and A/V hub 112 and other components in system 100 (suchas the one or more A/V display devices 114 and the one or more contentsources 116) may communicate using wireless and/or wired communication.During the wireless communication, these electronic devices maywirelessly communicate while: transmitting advertising frames onwireless channels, detecting one another by scanning wireless channels,establishing connections (for example, by transmitting associationrequests), and/or transmitting and receiving packets or frames (whichmay include the association requests and/or additional information aspayloads, such as user-interface information, device-state information,user-interface activity information, data, A/V content, etc.).

As described further below with reference to FIG. 23, portableelectronic device 110, A/V hub 112, the one or more A/V display devices114 and the one or more content sources 116 may include subsystems, suchas: a networking subsystem, a memory subsystem and a processorsubsystem. In addition, portable electronic device 110 and A/V hub 112,and optionally one or more of the one or more A/V display devices 114and/or the one or more content sources 116, may include radios 118 inthe networking subsystems. (Note that radios 118 may be instances of thesame radio or may be different from each other.) More generally,portable electronic device 110 and A/V hub 112 (and optionally one ormore of the one or more A/V display devices 114 and/or the one or morecontent sources 116) can include (or can be included within) anyelectronic devices with the networking subsystems that enable portableelectronic device 110 and A/V hub 112 (and optionally one or more of A/Vdisplay devices 114 and/or the one or more content sources 116) towirelessly communicate with each other. This wireless communication cancomprise transmitting advertisements on wireless channels to enableelectronic devices to make initial contact or detect each other,followed by exchanging subsequent data/management frames (such asassociation requests and responses) to establish a connection, configuresecurity options (e.g., Internet Protocol Security), transmit andreceive packets or frames via the connection, etc.

As can be seen in FIG. 1, wireless signals 120 (represented by a jaggedline) are transmitted from radio 118-1 in portable electronic device110. These wireless signals are received by at least A/V hub 112. Inparticular, portable electronic device 110 may transmit packets. Inturn, these packets may be received by a radio 118-2 in A/V hub 112.This may allow portable electronic device 110 to communicate informationto A/V hub 112. While FIG. 1 illustrates portable electronic device 110transmitting packets, note that portable electronic device 110 may alsoreceive packets from A/V hub 112.

In the described embodiments, processing of a packet or frame inportable electronic device 110 and A/V hub 112 (and optionally one ormore of the one or more A/V display devices 114 and/or the one or morecontent sources 116) includes: receiving wireless signals 120 with thepacket or frame; decoding/extracting the packet or frame from receivedwireless signals 120 to acquire the packet or frame; and processing thepacket or frame to determine information contained in the packet orframe (such as the information associated with a data stream). Forexample, the information from portable electronic device 110 may includeuser-interface activity information associated with a user interfacedisplayed on touch-sensitive display 124 in portable electronic device110, which a user of portable electronic device 110 uses to control A/Vhub 112, the one or more A/V display devices 114 and/or one of the oneor more content sources 116. (In some embodiments, instead of or inadditional to touch-sensitive display 124, portable electronic device110 includes a use interface with physical knobs and/or buttons that auser can use to control A/V hub 112, the one or more A/V display devices114 and/or one of the one or more content sources 116.) Alternatively,the information from A/V hub 112 may include device-state informationabout a current device state of one or more of A/V display devices 114or one of the one or more content sources 116 (such as on, off, play,rewind, fast forward, a selected channel, selected content, a contentsource, etc.), or may include user-interface information for the userinterface (which may be dynamically updated based on the device-stateinformation and/or the user-interface activity information).Furthermore, the information from A/V hub 112 and/or one of the one ormore content sources 116 may include audio and video that are displayedon one or more of A/V display devices 114, as well as displayinstructions that specify how the audio and video are to be displayed.(However, as noted previously, the audio and video may be communicatedbetween components in system 100 via wired communication. Therefore, asshown in FIG. 1, there may be a wired cable or link, such as ahigh-definition multimedia-interface (HDMI) cable 122, between A/V hub112 and A/V display device 114-1.)

Note that the communication between portable electronic device 110 andA/V hub 112 (and optionally one or more of the one or more A/V displaydevices 114 and/or the one or more content sources 116) may becharacterized by a variety of performance metrics, such as: a data rate,a data rate for successful communication (which is sometimes referred toas a ‘throughput’), an error rate (such as a packet error rate, or aretry or resend rate), a mean-square error of equalized signals relativeto an equalization target, intersymbol interference, multipathinterference, a signal-to-noise ratio, a width of an eye pattern, aratio of number of bytes successfully communicated during a timeinterval (such as 1-10 s) to an estimated maximum number of bytes thatcan be communicated in the time interval (the latter of which issometimes referred to as the ‘capacity’ of a channel or link), and/or aratio of an actual data rate to an estimated data rate (which issometimes referred to as ‘utilization’). Moreover, the performanceduring the communication associated with different channels may bemonitored individually or jointly (e.g., to identify dropped packets).

The communication between portable electronic device 110 and A/V hub 112(and optionally one or more of the one or more A/V display devices 114and/or the one or more content sources 116) in FIG. 1 may involve one ormore independent, concurrent data streams in different wireless channels(or even different communication protocols, such as different Wi-Ficommunication protocols) in one or more connections or links, which maybe communicated using multiple radios. Note that the one or moreconnections or links may each have a separate or different service setidentifier on a wireless network in system 100 (which may be aproprietary network or a public network). Moreover, the one or moreconcurrent data streams may, on a dynamic or packet-by-packet basis, bepartially or completely redundant to improve or maintain the performancemetrics even when there are transient changes (such as interference,changes in the amount of information that needs to be communicated,movement of portable electronic device 110, etc.), and to facilitateservices (while remaining compatible with the communication protocol,e.g., a Wi-Fi communication protocol) such as: channel calibration,determining of one or more performance metrics, performingquality-of-service characterization without disrupting the communication(such as performing channel estimation, determining link quality,performing channel calibration and/or performing spectral analysisassociated with at least one channel), seamless handoff betweendifferent wireless channels, coordinated communication betweencomponents, etc. These features may reduce the number of packets thatare resent, and, thus, may decrease the latency and avoid disruption ofthe communication and may enhance the experience of one or more users orviewers of content on the one or more A/V display devices 114.

As noted previously (and described further below with reference to FIG.6), a user may control A/V hub 112, one or more of the one or more A/Vdisplay devices 114 and/or one of the one or more content sources 116via the user interface displayed on touch-sensitive display 124 onportable electronic device. In particular, at a given time, the userinterface may include one or more virtual icons that allow the user toactivate, deactivate or change functionality or capabilities of A/V hub112, one or more of A/V display devices 114 and/or one or more ofcontent sources 116. For example, a given virtual icon in the userinterface may have an associated strike area on a surface oftouch-sensitive display 124. If the user makes and then breaks contactwith the surface (e.g., using one or more fingers or digits, or using astylus) within the strike area, portable electronic device 110 (such asa processor executing a program module) may receive user-interfaceactivity information indicating activation of this command orinstruction from a touch-screen input/output (I/O) controller, which iscoupled to touch-sensitive display 124. (Alternatively, touch-sensitivedisplay 124 may be responsive to pressure. In these embodiments, theuser may maintain contact with touch-sensitive display 124 with anaverage contact pressure that is usually less than a threshold value,such as 10-20 kPa, and may activate a given virtual icon by increase theaverage contact pressure with touch-sensitive display 124 above thethreshold value.) In response, the program module may instruct aninterface circuit in portable electronic device 110 to wirelesslycommunicate the user-interface activity information indicating thecommand or instruction to A/V hub 112, and A/V hub 112 may communicatethe command or the instruction to the target component in system 100(such as A/V display device 114-1). This instruction or command mayresult in A/V display device 114-1 turning on or off, displaying contentfrom a particular source, performing a trick mode of operation (such asfast forward, reverse, fast reverse or skip), etc.

One problem with using existing remote controls to control the operationof another component or entertainment device is that the remote controldoes not receive any feedback from the entertainment device. Forexample, many existing remote controls use infra-red communication.However, typically existing infra-red communication protocols areunidirectional or one-way communication, i.e., from a remote control tothe entertainment device. Consequently, the remote control usually doesnot have any knowledge of the effects of the commands or instructionsthat are communicated to the entertainment device. In particular, theremote control is typically unaware of a current state of theentertainment device, such as whether the entertainment device is in: apower-on state, a power-off state, a playback state, a trick-mode state(such as fast forward, fast reverse, or skip), a pause state, a standby(reduced-power) state, a record state, a state in which contentassociated with a given content source (such as cable television, asatellite network, a web page on the Internet, etc.) is received orprovided, and/or another state. (Note that one or more of the states maybe nested or concurrent with each other, such as the power-on state andthe playback state.) By operating blindly in this way, existing remotecontrol are unable to leverage knowledge of the current state of theentertainment device to improve the user experience.

This problem is addressed in system 100. In particular, as describedfurther below with reference to FIGS. 21 and 22, A/V hub 112 maydetermine the current state of one or more of the components in system100, such as the current state of A/V display device 114-1 and/or one ofthe one or more content sources 116. This device-state information maybe determined by A/V hub 112 using hardware and/or software, and A/V hub112 may determine the device-state information even for legacyentertainment devices that are only capable of receiving commands orinstructions (i.e., that are only capable of unidirectionalcommunication). For example, as described further below with referenceto FIGS. 21 and 22, whether or not a given component or entertainmentdevice in system 100 is electrically coupled to A/V hub 112 may bedetermined using a state-detection circuit that detects whether there iselectrical coupling between the entertainment device and an inputconnector to A/V hub 112 (such as an HDMI connector or port that can beelectrically coupled to HDMI cable 122). If the electrical coupling isdetected, the type of the given entertainment device (such as atelevision, a DVD player, a satellite receiver, etc.) and/or themanufacturer or provider of the given entertainment device may bedetermined by A/V hub 112 by providing a series of commands orinstructions to the given entertainment device (e.g., such as commandsor instructions that are specific to a particular type of entertainmentdevice, specific to a particular manufacturer, and/orconsumer-electronics-control commands in the HDMI standard orspecification), and then monitoring, as a function of time, changes in adata stream (as indicated by the number of packets or frames and/or thepayloads in the packets or frames) to and/or from the givenentertainment device to see if there was a response to a particularcommand or instruction. Moreover, the state-detection circuit maydetermine whether the given entertainment device is in the power-onstate or the power-off state by monitoring a voltage, a current and/oran impedance on, through or associated with one or more pins in theinput connector. Alternatively or additionally, A/V hub 112 maydetermine whether the given entertainment device is in the power-onstate or the power-off state by monitoring, as a function of time, thedata stream (as indicated by the number of packets or frames and/or thepayloads in the packets or frames) to and/or from the givenentertainment device. Similarly, A/V hub 112 may determine the currentstate of the given entertainment device, such as whether the givenentertainment device responded to a command or instruction that wasprovided to the given entertainment device by A/V hub 112, bymonitoring, as a function of time, changes in the data stream (asindicated by the number of packets or frames and/or the payloads in thepackets or frames) to and/or from the given entertainment device. Thus,the device-state information for the given entertainment devicedetermined by A/V hub 112 may include: presence or absence information(such as whether there is electrical coupling or a wireless connectionwith the given entertainment device), identity information (such as thetype of the given entertainment device and/or the manufacturer of thegiven entertainment device) and/or the current state.

Using the device-state information A/V hub 112 and/or portableelectronic device 110 may dynamically adapt the user interface displayedon touch-sensitive display 124 on portable electronic device 110. Forexample, A/V hub 112 may provide, via radio 118-2, device-stateinformation to portable electronic device 110 specifying a current stateof the given entertainment device. (Thus, this feedback technique mayinclude bidirectional or two-way communication between A/V hub 112 andportable electronic device 110.) After radio 118-1 receives thedevice-state information, portable electronic device 110 (such as aprogram module executed in an environment, e.g., an operating system, inportable electronic device 110) may generate a user interface thatincludes one or more virtual command icons associated with the currentstate and one or more related states of the given entertainment device.Note that the one or more related states may be related to the currentstate in a state diagram (which may be stored in memory in portableelectronic device 110) by corresponding operations that transition thegiven entertainment device from the current state to the one or morerelated states. Then, portable electronic device 110 may display theuser interface on touch-sensitive display 124.

In some embodiments, A/V hub 112 provides information specifying thetype of the given entertainment device, the manufacturer of the givenentertainment device, and/or context information that specifies acontext of content (such as A/V content) displayed on the entertainmentdevice (such as A/V display device 114-1). For example, the context mayinclude a type of the content (such as sports, television, a movie,etc.), a location in the content (such as a timestamp, an identifier ofa sub-section in the content and/or a proximity to a beginning or an endof the content), etc. In these embodiments, the one or more virtualcommand icons (and, thus, the user interface) may be based on the typeof the given entertainment device, the manufacturer and/or the context.Thus, only virtual command icons that are relevant to the givenentertainment device, the manufacturer and/or the context may beincluded in the user interface.

Moreover, when the user activates one of the virtual command icons inthe user interface, the touch-screen I/O controller in portableelectronic device 110 may provide user-interface activity informationspecifying activation of a virtual command icon in the one or morevirtual command icons, where the activation of the virtual command iconspecifies a transition of the given entertainment device from thecurrent state to a new current state in the state diagram. As notedpreviously, the activation of the virtual command icon may involve auser of portable electronic device 110 contacting touch-sensitivedisplay 124 within a strike area of the virtual command icon and thenreleasing the contact. In response to receiving the user-interfaceactivity information, portable electronic device 110 may: modify theuser interface to change the one or more virtual command icons based onthe new current state; and display the modified user interface ontouch-sensitive display 124. Note that portable electronic device 110may wait to change the one or more virtual command icons until thedevice-state information received from A/V hub 112 indicates that thegiven entertainment device has transitioned to the new current state inresponse to a command or an instruction associated with the activationof the one of the virtual command icons. Thus, portable electronicdevice 110 may repeatedly perform the generating and the displayingoperations so that the user interface is dynamically updated as thecurrent state changes.

Alternatively or additionally, instead of portable electronic device 110generating the user interface, A/V hub 112 may generate user-interfaceinformation that specifies the user interface (or instructionsspecifying objects or graphical information in the user interface) basedon the one or more related states in the state diagram (which may bestored in memory in A/V hub 112) and one or more of: the device-stateinformation, the type of the given entertainment device, themanufacturer of the given entertainment device, the context,user-interface activity information specifying activation (by the user)of one of the virtual command icons in the user interface (which may bereceived, via radios 118, from portable electronic device 110), and/or adisplay format in portable electronic device 110. Then, A/V hub 112 mayprovide, via radios 118, the user-interface information to portableelectronic device 110 for display on touch-sensitive display 124.

In this way, the user interface may be dynamically updated as thecomponents in system 100 respond to commands or instructions receivedfrom portable electronic device 110 and/or A/V hub 112, so that thecurrently relevant one or more virtual icons are included in the userinterface. This capability may simplify the user interface and make iteasier for the user to navigate through and/or use the user interface.

Moreover, as described further below with reference to FIGS. 18 and 19,the device-state information may be used by A/V hub 112 to dynamicallyadapt or change a number of operations needed or used to cast HDMIcontent from one A/V display device to another A/V display device, suchas from A/V display device 114-1 to A/V display device 114-2 (which maybe in the same room or a different room in the environment that includesA/V hub 112). (While HDMI content is used as an illustrative example, inother embodiments content that is compatible with another format orstandard is used in the embodiments of the display technique, including:H.264, MPEG-2, a QuickTime video format, MPEG-4 and/or MP4. Moreover,the video mode of the content may be 720p, 1080i, 1080p, 1440p, 2000,2160p, 2540p, 4000p and/or 4320 p) In particular, in response touser-interface activity information received from portable electronicdevice 110 that specifies user instruction to cast HDMI contentassociated with a content source (such as content source 116-1) from A/Vdisplay device 114-1 to A/V display device 114-2 (such as useractivation of a virtual command icon in a user interface), A/V hub 112may access device-state information that specifies a current state ofA/V display device 114-2. Then, A/V hub 112 may perform the casting ofthe HDMI content from A/V display device 114-1 to A/V display device114-2 (and, in particular, from the content source to A/V display device114-2) using a dynamic number of operations based on the device-stateinformation, so that the HDMI content is displayed on a display in A/Vdisplay device 114-2. For example, when the device-state informationindicates that A/V display device 114-2 is powered on, a power-onoperation may be excluded from the dynamic number of operations.Alternatively or additionally, when the device-state informationindicates that A/V display device 114-2 is set to receive the HDMIcontent from content source 116-1, an operation to set A/V displaydevice 114-2 to receive the HDMI content from content source 116-1 maybe excluded from the dynamic number of operations. Thus, based ondevice-state awareness, the display technique may adapt or change thedynamic number of operations so that operations such as casting areperformed efficiently (i.e., are less time-consuming).

Furthermore, as described further below with reference to FIGS. 2 and 3,in response to user-interface activity information received fromportable electronic device 110 that specifies a user selection of acontent source (such as content source 116-1), A/V hub 112 may provide arequest for HDMI content to content source 116-1 based on the userselection. When A/V hub 112 receives the HDMI content from contentsource 116-1, A/V hub 112 may provide the HDMI content and displayinstructions to A/V display device 114-1 as frames with the HDMI contentare received from content source 116-1 (e.g., in real time), so that theHDMI content is displayed in a tiled window on the display in A/Vdisplay device 114-1 while other HDMI content from another contentsource (such as content source 116-2) is displayed on the display. Notethat a ‘tiled window’ may include window in at least a portion of thedisplay area that does not overlap another window, but that presents theHDMI content overlaid on a background of the window. (Alternatively oradditionally, the HDMI content may be displayed in a cascaded oroverlapped window that at least partially overlaps or that is partiallyoverlapped by another window.) Thus, the display technique may allowHDMI content to be presented in a tiled window in real time (such asframes with the HDMI content are received from content source 116-1).

In some embodiments, as described further below with reference to FIGS.4 and 5, A/V hub 112 determines display instructions specifyinginformation to be displayed on the display in an A/V display device(such as A/V display device 114-1) that includes icons associated withcontent sources (such as content sources 116). Then, A/V hub 112provides the display instructions to A/V display device 114-1 fordisplay on the display. For example, the information to be displayed mayspecify content (such as HDMI content) that is displayed on the displayand where the content is displayed (such as one or more tiled windows).Therefore, the information to be displayed may be based on a format ofthe display, such as: a display size, display resolution, display aspectratio, display contrast ratio, a display type, etc. Then, in response toreceiving a user selection of content source 116-1 (or user-interfaceactivity information that specifies the user selection) based onactivation of a single command feature in portable electronic device 110that is associated with content source 116-1 (such as a virtual commandicon in a user interface displayed on portable electronic device 110),A/V hub 112 may provide a request for HDMI content to content source116-1. When A/V hub 112 receives the HDMI content from content source116-1, A/V hub 112 may provide the HDMI content and second displayinstructions to A/V display device 114-1 as frames with the HDMI contentare received from content source 116-1, so that the HDMI content isdisplayed in a tiled window on the display in A/V display device 114-1.For example, the information to be displayed may specify that the HDMIcontent is displayed on the display in a new tiled window. Thus, thedisplay technique may allow the new tiled window to be created in thedisplay based on selection or activation of the single command featurewithout requiring further action or additional operations be performedby the user.

Additionally, as described further below with reference to FIGS. 14-17,A/V hub 112 may determine the display instructions specifying theinformation to be displayed on the display in an A/V display device(such as A/V display device 114-1) that includes icons associated withcontent sources 116. These content sources may be located at arbitraryor different locations in a structure (such as one or more rooms in abuilding), including locations external to an environment (such as atleast a portion of a room in the building, which may be different thanthe one or more rooms) that includes A/V hub 112 and A/V display device114-1. Then, A/V hub 112 provides the display instructions to A/Vdisplay device 114-1 for display on the display. Thus, leveraging thewired and/or wireless network(s) associated with A/V hub 112, thedisplay technique may allow a wide variety of content sources providedby different providers, and which are locally or remoted located, to bedisplayed on A/V display device 114-1.

In these ways, the display technique may allow a user to control the A/Vhub, one or more A/V display devices and/or one or more content sourceswith less effort (such as in less time, with fewer operations and/orwith less confusion), and to do so with fewer errors or mistakes.Consequently, the display technique may reduce user frustration and/ormay improve the user experience when using the portable electronicdevice, the A/V hub, the one or more A/V display devices and/or the oneor more content sources.

Although we describe the network environment shown in FIG. 1 as anexample, in alternative embodiments, different numbers or types ofelectronic devices may be present. For example, some embodimentscomprise more or fewer electronic devices. As another example, inanother embodiment, different electronic devices are transmitting and/orreceiving packets or frames. While portable electronic device 110 andA/V hub 112 are illustrated with a single instance of radios 118, inother embodiments portable electronic device 110 and A/V hub 112 (andoptionally one or more of A/V display device 114-1 and/or the one ormore content sources 116) may include multiple radios.

We now describe embodiments of a display technique. FIG. 2 presents aflow diagram illustrating method 200 for providing HDMI content, whichmay be performed by an A/V hub, such as A/V hub 112 (FIG. 1). Duringoperation, the A/V hub (such as a control circuit or control logic,e.g., a processor executing a program module and/or or a circuit)receives, via an interface circuit in the A/V hub, user-interfaceactivity information (operation 210) from a portable electronic device,where the user-interface activity information specifies user selectionof a content source. For example, the user selection may involveactivation of a virtual command icon associated with the content source,which is displayed in a user interface on the portable electronicdevice. Alternatively, the user selection may involve activation of aphysical button associated with the content source, which is included inthe portable electronic device. Note that the A/V hub may optionally:generate user-interface information that specifies the user interfacethat includes one or more virtual command icons, including the virtualcommand icon, which are associated with one or more content sources; andprovides the user-interface information to the portable electronicdevice for display on a touch-sensitive display in the portableelectronic device. However, in other embodiments the user-interfaceinformation is optionally generated by the portable electronic device.

Then, the A/V hub provides, via the interface circuit, a request forhigh-definition multimedia-interface (HDMI) content (operation 212) tothe content source based on the user selection. Moreover, the A/V hubreceives, via the interface circuit, the HDMI content (operation 214)from the content source, and the A/V hub may optionally determine thedisplay instructions (operation 216) based on a format of the display.The request from the A/V hub in response to the activation of thevirtual command icon or the physical button may include one or morecommands or instructions for the content source, including selectivelyactivating or turning on the content source (if the content source isnot already activated or turned on) and/or specifying the HDMI content.Therefore, in some embodiments, the request may be based, at least inpart, on device-state information about the content source.

Next, the A/V hub provides, via the interface circuit, the HDMI contentand display instructions (operation 218) to an A/V display device asframes with the HDMI content are received from the content source, sothat the HDMI content is displayed on a display in the A/V displaydevice. For example, providing the HDMI content and the displayinstructions may involve transcoding the HDMI content based on a formatof the display. Note that the display instructions specify that the HDMIcontent is to be displayed in a tiled window on the display while otherHDMI content from another content source is displayed on the display.

In some embodiments, the A/V hub optionally performs one or moreadditional operations (operation 220). For example, the A/V hub mayoptionally: receive, via the interface circuit, the other HDMI contentfrom the other content source; and provide, via the interface circuit,the other HDMI content to the A/V display device, so that the other HDMIcontent is displayed on the display. Moreover, the A/V hub mayoptionally provide second display instructions that specify how theother HDMI content is to be displayed on the display. Furthermore, theA/V hub may optionally determine the second display instructions basedon a format of the display.

Note that method 200 may address security issues associated with theconcurrent display of the tiled window(s). For example, the HDMI contentmay be displayed in the tiled window on the display while the other HDMIcontent from the other content source is displayed on the displaywithout a risk of hacking (such as intentional communication ofmalicious content, such as malicious software, malware or a so-called‘virus’).

FIG. 3 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112 andA/V display device 114-1. In particular, processor 310 may optionallygenerate user-interface information 312 that specifies a user interfacethat includes one or more virtual command icons, including a virtualcommand icon, which are associated with content sources 116. Then,processor 310 may optionally provide, via interface circuit 314,user-interface information 312 to portable electronic device 110. Afterreceiving user-interface information 312, portable electronic device 110may optionally display associated user interface 316 on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 312 is optionally generated by portableelectronic device 110.

For example, interface circuit 318 in portable electronic device 110 mayreceive user-interface information 312, which is then provided toprocessor 320. Alternatively, processor 320 may generate user-interfaceinformation 312. Then, based on user-interface information 312,processor 320 may provide information specifying user interface 316 totouch-sensitive input/output (I/O) controller 322, which provides theinformation specifying user interface 316 to touch-sensitive display124.

Moreover, touch-sensitive display 124 may provide to informationspecifying user interaction 324 to touch-sensitive I/O controller 322.In turn, touch-sensitive I/O controller 322 may interpret theinformation specifying user interaction 324 to determine user-interfaceactivity information 326. For example, user-interface activityinformation 326 may specify user selection of one of content sources116, such as user activation of the virtual command icon associated withthe one of content sources 116. Touch-sensitive I/O controller 322 mayprovide user-interface activity information 326 to processor 320, whichmay provide user-interface activity information 326 to interface circuit318.

Next, portable electronic device 110 (such as via interface circuit 318)may provide user-interface activity information 326 to A/V hub 112.After receiving user-interface activity information 326, interfacecircuit 314 may provide user-interface activity information 326 toprocessor 310. In response, processor 310 may instruct interface circuit314 to provide request 328 for HDMI content 330 to the one of contentsources 116. In addition, processor 310 may optionally determine displayinstructions 332 based on a format of a display in A/V display device114-1. Alternatively, display instructions 332 may be predetermined orpredefined.

After receiving request 328, the one of content sources 116 may provideHDMI content 330 to A/V hub 112. Next, interface circuit 314 may provideHDMI content 330 and/or display instructions 332 (which may be providedifferentially when there or changes or regularly, such in each packetor in one of every N packets) to A/V display device 114-1 as frames withHDMI content 330 are received from the one of content sources 116, sothat HDMI content 330 is displayed on the display in A/V display device114-1. (Alternatively, in some embodiments interface circuit 314provides HDMI content 330 to processor 310, which instructs interfacecircuit 314 to provide HDMI content 330 and display instructions 332 toA/V display device 114-1 as frames with HDMI content 330 are receivedfrom the one of content sources 116.)

Note that display instructions 332 may specify that HDMI content 330 isto be displayed in a tiled window on the display while additional HDMIcontent from another of content sources 116 is displayed on the display.For example, A/V hub 112 may optionally: receives, via interface circuit314, the additional HDMI content from the other of content sources 116;and provides, via interface circuit 314, the additional HDMI content toA/V display device 114-1, so that additional HDMI content is displayedon the display. Moreover, processor 310 may optionally determineadditional display instructions that specify how the additional HDMIcontent is to be displayed on the display based on a format of thedisplay. Furthermore, interface circuit 314 may optionally provide theadditional display instructions to A/V display device 114-1.

The user may create, modify or discontinue a tiled window by activatinga single command feature in a user interface displayed on the portableelectronic device. This is shown in FIG. 4, which presents a flowdiagram illustrating a method 400 for providing HDMI content. Note thatthis method may be performed by an A/V hub, such as A/V hub 112 (FIG.1). During operation, the A/V hub (such as a control circuit or controllogic, e.g., a processor executing a program module and/or or a circuit)determines display instructions (operation 410) specifying informationto be displayed on a display in an A/V display device, where theinformation to be displayed includes icons associated with contentsources. Then, the A/V hub provides, via an interface circuit in the A/Vhub, the display instructions (operation 412) to the A/V display devicefor display on the display. Moreover, the A/V hub receives, via theinterface circuit, user-interface activity information (operation 414)from a portable electronic device, where the user-interface activityinformation specifies user selection of a content source based onactivation of a single command feature in the portable electronic devicethat is associated with the content source. For example, the singlecommand feature may include a virtual command icon displayed in the userinterface on a touch-sensitive display in the portable electronicdevice. Alternatively, the single command feature may include a physicalbutton. Note that the A/V hub may optionally: generate user-interfaceinformation that specifies the user interface that includes one or morevirtual command icons, including the virtual command icon, which areassociated with the content sources; and provides the user-interfaceinformation to the portable electronic device for display on thetouch-sensitive display in the portable electronic device. However, inother embodiments the user-interface information is optionally generatedby the portable electronic device.

Next, the A/V hub provides, via the interface circuit, a request forHDMI content (operation 416) to the content source based on the userselection, and receives, via the interface circuit, the HDMI content(operation 418) from the content source. In some embodiments, the A/Vhub optionally determines second display instructions (operation 420)based on a format of the display. The request from the A/V hub inresponse to the activation of the virtual command icon or the physicalbutton may include one or more commands or instructions for the contentsource, including selectively activating or turning on the contentsource (if the content source is not already activated or turned on)and/or specifying the HDMI content. Therefore, in some embodiments, therequest may be based, at least in part, on device-state informationabout the content source.

Furthermore, the A/V hub provides the HDMI content and the seconddisplay instructions (operation 422) to the A/V display device as frameswith the HDMI content are received from the content source, so that theHDMI content is displayed on a display in the A/V display device, wherethe second display instructions specify that the HDMI content is to bedisplayed in a tiled window on the display. For example, providing theHDMI content and the second display instructions may involve transcodingthe HDMI content based on a format of the display.

Additionally, the A/V hub may optionally perform one or more additionaloperations (operation 424). For example, the A/V hub may: receive, viathe interface circuit, second user-interface activity information fromthe portable electronic device, where the second user-interface activityinformation specifies user de-activation of the content source based onactivation of the single command feature in the portable electronicdevice that is associated with the content source; provide, via theinterface circuit, a request to discontinue the HDMI content to thecontent source; and provide third display instructions to the A/Vdisplay device, so that display of the tiled window on the display isdiscontinued.

Alternatively or additionally, the A/V hub may: receive, via theinterface circuit, third user-interface activity information from theportable electronic device, where the third user-interface activityinformation specifies user selection of a second content source based onactivation of a second single command feature in the portable electronicdevice that is associated with the second content source; provide, viathe interface circuit, a request for second HDMI content to the secondcontent source; receive, via the interface circuit, the second HDMIcontent from the second content source; and provide the second HDMIcontent and third display instructions to the A/V display device asframes with the second HDMI content are received from the second contentsource, so that the second HDMI content is displayed on the display inthe A/V display device concurrently with the HDMI content in the titledwindow. Note that the third display instructions specify that the secondHDMI content is to be displayed in a second tiled window on the display.

Furthermore, subsequently, the A/V hub may receive, via the interfacecircuit, fourth user-interface activity information from the portableelectronic device that specifies user selection of one of the tiledwindow and the second tiled window. In response, the A/V hub may providefourth display instructions to the A/V display device, so that givenHDMI content, which is one of the HDMI content and the second HDMIcontent, is displayed in a central tiled window on the display anddisplay of a given tiled window, which is one of the tiled window andthe second tiled window, is discontinued.

FIG. 5 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay device 114-1 and one or more content sources 116. In particular,processor 310 may determine display instructions 510 specifyinginformation to be displayed on a display in an A/V display device 114-1.Then, processor 310 may optionally provide, via interface circuit 314,display instructions 510 to A/V display device 114-1 for display on thedisplay.

Moreover, processor 310 may optionally provide, via interface circuit314, user-interface information 512 to portable electronic device 110.After receiving user-interface information 512, portable electronicdevice 110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 512 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide user-interface activityinformation 514 that specifies user selection of one of content sources116 (such as user activation of a single command feature associated withthe one of content sources 116) to A/V hub 112. After receivinguser-interface activity information 514, interface circuit 314 mayprovide user-interface activity information 514 to processor 310. Inresponse, processor 310 may instruct interface circuit 314 to providerequest 516 for HDMI content 518 to the one of content sources 116. Inaddition, processor 310 may optionally determine display instructions520 based on a format of the display in A/V display device 114-1.Alternatively, display instructions 520 may be predetermined orpredefined.

After receiving request 516, the one of content sources 116 may provideHDMI content 518 to A/V hub 112. Furthermore, interface circuit 314 mayprovide HDMI content 518 and display instructions 520 to A/V displaydevice 114-1 as frames with HDMI content 518 are received from the oneof content sources 116, so that HDMI content 518 is displayed on thedisplay in A/V display device 114-1. Note that display instructions 520may specify that HDMI content 518 is to be displayed in a tiled windowon the display while additional HDMI content from another of contentsources 116 is displayed on the display. (Alternatively, in someembodiments interface circuit 314 provides HDMI content 518 to processor310, which instructs interface circuit 314 to provide HDMI content 518and display instructions 1520 to A/V display device 114-1 as frames withHDMI content 518 are received from the one of content sources 116.)

In response to receiving, via interface circuit 314, user-interfaceactivity information 522 from portable electronic device 110, processor310 may optionally determine and provide display instructions 524 to A/Vdisplay device 114-1 and/or optional command 526 to the one of contentsources 116. For example, if user-interface activity information 522specifies de-activation of the one of content sources 116 based on useractivation of the single command feature (or another single commandfeature) in portable electronic device 110, display instructions 524 mayindicate that the tiled window on the display is discontinued. Inaddition, processor 310 may provide optional command 526 to the one ofcontent sources 116 to discontinue HDMI content 518. Alternatively,user-interface activity information 522 may specify user selection ofthe tiled window, and display instructions 524 may indicate that HDMIcontent 518 is displayed in a central tiled window on the display anddisplay of the tiled window on the display is discontinued.

In some embodiments, user-interface activity information 522 specifiesuser selection of another one of content sources 116 (such as useractivation of another single command feature associated with the otherone of content sources 116). In these embodiments, optional command 526requests additional HDMI content from the other one of content sources116. As the additional HDMI content is received by A/V hub 112,interface circuit 314 may provide the additional HDMI content anddisplay instructions 524 to A/V display device 114-1 as frames with theadditional HDMI content are received from the one of content sources116, so that the additional HDMI content is displayed on the display inA/V display device 114-1. Note that display instructions 524 may specifythat the additional HDMI content is to be displayed in another tiledwindow on the display while HDMI content 518 from the one of contentsources 116 is displayed on the display in the tiled window.

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In the process, the display technique may reduce userfrustration, and thus may improve user satisfaction when using theportable electronic device, the A/V hub, the A/V display device and/orthe one or more content sources.

In an exemplary embodiment, the portable electronic device includes atouch-sensitive display and the user interface includes features (suchas virtual command icons) associated with the content sources and thelayout of the content displayed on the A/V display device. This userinterface is shown in FIG. 6, which presents a drawing illustrating auser interface 600 on a touch-sensitive display 124 in portableelectronic device 110 in FIG. 1. In particular, user interface 600 mayinclude one or more virtual command icons 610 that are associated withcontent sources (such as content sources 116 in FIG. 1). By activatingone of virtual command icons 610 (such as by using a finger or digit totouch and then break contact with a surface of the touch-sensitivedisplay within a strike area associated with the virtual command icon),a user may provide user-interface activity information that instructs anA/V hub (such as A/V hub 112 in FIG. 1) to request HDMI content from theone of the content sources (such as the DVR) and, after receiving theHDMI content, to provide it along with display instructions to an A/Vdisplay device (such as one of A/V display devices 114 in FIG. 1). Asnoted previously, the HDMI content may be presented on a display in theA/V display device in a tiled window in real-time (i.e., as frames ofthe HDMI content are received from the content source). Note that theA/V hub may collect the HDMI content in a buffer until a frame isreceived, and then the A/V hub may provide the complete frame to the A/Vdisplay device. Alternatively, the A/V hub may provide packets withportions of a frame to the A/V display device as they are received.

In some embodiments, if the user makes and maintains contact with thesurface of the touch-sensitive display within a strike area associatedwith one of virtual command icons 610 or if the user activates thisvirtual command icon, the resulting user-interface activity informationfrom the portable electronic device instructs the A/V hub to providedisplay instructions for a list of display options to be displayed onthe A/V display device. In addition, if the user makes and maintainscontact with the surface of the touch-sensitive display within a strikearea associated with one of virtual command icons 610 or if the useractivates this virtual command icon, an associated one of virtualcommand icons 612 may be displayed. In particular, user interface 600may be modified (either by the portable electronic device and/or inresponse to user-interface information received from the A/V hub) toinclude one of virtual command icons 612 associated with a displayoption. For example, the display option may include: discontinuing thedisplay of the HDMI content from the content source (and, thus, todiscontinue or close the tiled window), as indicated by the circle withthe ‘X’ for virtual command icon 612-1; and/or, if the one of virtualcommand icons 612 associated with a new content source (i.e., a contentsource whose HDMI content is not currently displayed), displaying theHDMI content from the new content source in another tiled window, asindicated by the circles with the ‘+’ for virtual command icon 612-2. Inparticular, if the user makes and maintains contact with the surface ofthe touch-sensitive display within a strike area associated with virtualcommand icon 610-1 or if the user activates this virtual command icon,virtual command icon 612-1 may be displayed. Thus, if the user touchesvirtual command icon 610-1, the corresponding one of virtual commandicons 612 (the circle with the ‘X’) may be displayed. Then, if the userslides the contact area over virtual command icon 612-1 and releases thecontact (thereby providing an activation command), the portableelectronic device providing user-interface activation information to theA/V hub that instructs the A/V hub to communicate with the DVR todiscontinue the data stream with content (such as HDMI content) from theDVR. Alternatively, if the user makes and maintains contact with thesurface of the touch-sensitive display within a strike area associatedwith virtual command icon 610-2 or if the user activates this virtualcommand icon, virtual command icon 612-2 may be displayed. Thus, if theuser touches virtual command icon 610-2, the corresponding one ofvirtual command icons 612 (the circle with the ‘+’) may be displayed.Then, if the user slides the contact area over virtual command icon612-2 and releases the contact (thereby providing an activationcommand), the portable electronic device providing user-interfaceactivation information to the A/V hub that instructs the A/V hub tocommunicate with the Blue-Ray player to initiate a data stream withcontent (such as HDMI content) that will be displayed on the A/V displaydevice (such as in a tiled window). Note, therefore, that virtualcommand icons 612 may be dynamically displayed in the user interfacedepending on which virtual command icon 610 is selected, and may dependupon the current state of the associated content source, as well aswhether at least one other content source is currently providing a datastream with content (such as HDMI content).

Note that the content sources may be associated with different providersand may be at arbitrary or different locations in an environment (suchas in different rooms in a structure than the A/V hub and the A/Vdisplay device). Thus, the display technique may be independent of orinsensitive to the actual location of the content source(s) that providethe HDMI content. In these embodiments, if the user makes and maintainscontact with the surface of the touch-sensitive display within a strikearea associated with one of virtual command icons 610 and/or 612, userinterface 600 may be modified (either by the portable electronic deviceand/or in response to user-interface information received from the A/Vhub) to include information specifying the location of an associatedcontent source in the environment (such as in the information displayedin one of virtual command icons 610). However, in other embodimentsvirtual command icons 610 indicates available content sources to theuser without indicating the locations of the content sources. Thus, thedisplay technique may allow the portable electronic device and the A/Vhub to operate in a content-source-location independent manner, i.e.,the associated HDMI content may be displayed independently of a locationof the associated content source.

Similarly, as described further below with reference to FIGS. 18-20, theuser may use virtual command icon 614 in user interface 600 to performone-operation casting of content (such as HDMI content) from one A/Vdisplay device in the environment to another A/V display device in theenvironment. In particular, the user may be able to cast the HDMIcontent by activating virtual command icon 614. For example, if the userselects a content source that is currently providing a data stream to anA/V display device somewhere in system 100 (FIG. 1) by touching thesurface of the touch-sensitive display within a strike area associatedwith virtual command icons 610, user interface 600 may be modified(either by the portable electronic device and/or in response touser-interface information received from the A/V hub) to include virtualcommand icon 614. If the data stream with content from the selectedcontent source is currently being displayed on an A/V display device atanother location (such as in a different room), then, if the useractivates virtual command icon 614, another data stream with thiscontent may be cast and displayed on another A/V display device, such asan A/V display device that the user is controlling using the portableelectronic device. This ‘one-step casting’ (based on the activation of asingle command feature, such as a single virtual command icon orphysical button) may involve the A/V hub providing one or more commandsor instructions in response to receiving information specifying anactivation command associated with the activation of virtual commandicon 614. In some embodiments, the one or more commands includeselectively turning on the other A/V display device (if the other A/Vdisplay device is not already activated), instructing the content sourceto provide the other data stream with the selected content to the A/Vhub, etc. Thus, the one-step casting may be performed based ondevice-state information, such as device-state information about theother A/V display device.

Alternatively, the user selects a content source that is currentlyproviding a data stream to an A/V display device at the user's currentlocation, then user interface 600 may be modified (either by theportable electronic device and/or in response to user-interfaceinformation received from the A/V hub) to include information specifyingavailable A/V display devices or information associated with theavailable A/V display devices (such as locations in the environment ofthe A/V display devices). Next, if the user activates one of the virtualcommand icons associated with one of the available A/V display devices(such as a virtual command icon associated with a location, e.g., atelevision in the living room or a bedroom), the resultinguser-interface activity information from the portable electronic devicemay instruct the A/V hub to provide display instructions to thespecified A/V display device and to route or cast the HDMI content fromone of the content sources to the specified A/V display device. Onceagain, this may involve one or more commands or instructions from theA/V hub, and the one or more commands or instructions may be selectedintelligently based on device-state information. In this way, the usermay simply and intuitively cast content, such as HDMI content, from oneA/V display device to another A/V display device.

We now describe embodiments of the information displayed on the A/Vdisplay device during methods 200 (FIG. 2) and/or 400 (FIG. 4). FIG. 7presents a drawing illustrating a display 700 (which is sometimesreferred to as a ‘command window’) in an A/V display device (such as A/Vdisplay device 114-1 in FIG. 1). In particular, display 700 shows a listof available content sources 710. However, currently no HDMI content isdisplayed on display 700.

When the user activates one of the virtual command icons associated withone of content sources 710 (such as DVR), the A/V hub may providedisplay instructions and, after requesting and receiving HDMI contentfrom the one of content sources 710, may provide the HDMI content to A/Vdisplay device. As shown in FIG. 8, which presents a drawingillustrating a display 800 in the A/V display device, the HDMI contentmay be presented in a tiled window 810. Note that the list of availablecontent sources highlights the DVR under ‘current and that the remainingcontent sources are shown below the horizontal line in alphabeticalorder. Thus, the user may use the user interface on the portableelectronic device to dynamic select and set up a tiling window using onestep or operation. (However, in other embodiments, more than one step oroperation may be used.) Note that if the virtual command icon for theDVR is still selected, the user may be presented with an ‘X’ virtualcommand icon that the user can use to discontinue the display of theHDMI content in tiled window 810. Thus, one or more of virtual commandicons 612 (FIG. 6) may be dynamically displayed on display 810 based onthe user's selections and/or activation of virtual command iconsconcurrently displayed on the user interface in the portable electronicdevice.

Similarly, when the user selects another of the virtual command iconsassociated with a second one of the content sources (such as a Blue-Rayplayer) and then activates a ‘+’ virtual command icon that isdynamically displayed next to the selected other virtual command icon,the A/V hub may provide display instructions and, after requesting andreceiving additional HDMI content from the second content source, mayprovide the additional HDMI content to A/V display device. As shown inFIG. 9, which presents a drawing illustrating a display 900 in the A/Vdisplay device, the additional HDMI content may be presented in a tiledwindow 910. Note that the additional HDMI content may be presented in atiled window 910 may be presented on display 900 concurrently orsimultaneously with the HDMI content presented in tiled window 810.Thus, there may be multiple independent real-time or live video streamspresented or displayed in display 900. The user may user the userinterface on the portable electronic device to select the audio for oneof tiled windows 810 and 910, which is then output on the portableelectronic device (such as to headphones), while the audio from theother of tiled windows 810 and 910 may be output on speakers in the sameroom or another room in the environment that includes the A/V hub.

While FIG. 9 illustrates tiled windows 810 and 910 arranged along avertical direction 912, in other embodiments different spatialarrangements or configurations of tiled windows 810 and 910 are used.This is illustrated in FIG. 10, which presents a drawing illustrating adisplay 1000 in the A/V display device in which tiled windows 810 and910 are arranged along a horizontal direction 1010.

Furthermore, as discussed previously, the user may use the userinterface on the portable electronic device to discontinue the displayof HDMI content from one of the content sources. For example, aftertiled window 810 is displayed, if the user holders their finger incontact with the surface of the touch-sensitive display on the portableelectronic device within the strike area of a virtual command iconassociated with the DVR, the user interface on the portable electronicdevice and the information to be displayed on the display in the A/Vdisplay device may be modified to indicate that the user has the optionto discontinue the display of HDMI content from the one of the contentsources (such as the DVR). In particular, a virtual command iconrepresented by an ‘X’ in a circle may be displayed adjacent to thevirtual command icon for the DVR. If the user activates this virtualcommand icon, the A/V hub may provide display instructions to the A/Vdisplay device to discontinue tiled widow 810 and may instruct the DVRto discontinue providing the HDMI content. As shown in FIG. 11, whichpresents a drawing illustrating a display 1100 in the A/V displaydevice, tiled window 810 may be discontinued, so that only tiled window910 with the additional HDMI content is displayed.

In some embodiments, the user may use the user interface on the portableelectronic device to move the HDMI content displayed in a tiled windowto a central tiled window in the display in the A/V display device. Thisis illustrated in FIG. 12, which presents a drawing illustrating adisplay 1200 in the A/V display device. As shown in FIG. 12, initiallythere may be one or more tiled windows 1210 that display HDMI contentassociated with one or more of the content sources that are currentlyproviding data streams with HDMI content, such as the DVR and thestreaming TV/media player. Initially, tiled window 1210-1 may displaythe HDMI content from the DVR, and tiled window 1210-2 may display theHDMI content from the streaming TV/media player. Note that when the usercontacts the surface of the touch-sensitive display within a strike areaof the virtual command icon for the DVR and/or activates this virtualcommand icon, an additional virtual command icon (as represented by thecircle with the ‘X’) that allows the user to discontinue the displayingof tiled window 1210-1 is dynamically displayed.

Then, if the user slides over and activates the virtual command icon forthe streaming TV/media player (by contacting and releasing the surfaceof the touch-sensitive display within an associated strike area) thepositions of tiled windows 1210 may be reversed. (Note that anadditional virtual command icon that allows the user to discontinue thedisplaying of tiled window 1210-2 is also dynamically displayed.) Inparticular, in response to activating the virtual command icon for thestreaming TV/media player, the A/V hub may provide display instructionsthat transitions the associated HDMI content to a central tiled windowand discontinues the initial tiled window. The result is shown in FIG.13, which presents a drawing illustrating a display 1300 with the HDMIcontent presented or displayed in central tiled window 1210-2. Thus, theuser may use the user interface on the portable electronic device todynamically change the HDMI content displayed in the central tiledwindow.

While the preceding discussion illustrated the use of a virtual commandicons in a user interface displayed on a touch-sensitive display in theportable electronic device, in other embodiments the user may activatephysical buttons. For example, the user may point the portableelectronic device at a location on the A/V display device and may pressa button. The location on the A/V display device, which may correspondto a virtual command icon, may be identified based on an infra-redsignal from the portable display device, wireless ranging and/ororientation of the portable display device, etc. This user-interfaceinformation may be used by the A/V hub to perform one or more commandsor instructions and, as needed, to modify the displayed user interfaceon the A/V display device.

In some embodiments, tiled windows with video content are displayedwithout the user activating an associated virtual command icon or aphysical button. For example, the display of a tiled window may be eventdriven, such as when a security or intrusion-detection device is trigger(such as based on sound or motion detection) or when a baby monitordetects motion or a baby crying. In these cases, a tiled window withvideo from a security camera or a baby camera may be automaticallydisplayed based on the context. This may allow the user to visuallyassess whether further action is required.

Embodiments of the display technique are further illustrated in FIG. 14,which presents a flow diagram illustrating a method 1400 for providingdisplay instructions. This method may be performed by an A/V hub, suchas A/V hub 112 (FIG. 1). During operation, the A/V hub (such as acontrol circuit or control logic, e.g., a processor executing a programmodule and/or or a circuit) generates display instructions (operation1410) specifying information to be displayed on a display in an A/Vdisplay device, where the information to be displayed includes iconsassociated with the content sources. Note that the content sources maybe located at arbitrary or different locations in a structure, includinglocations external to an environment of the A/V hub and the A/V displaydevice. For example, the environment may include one or more rooms inthe structure, and the arbitrary or different locations may be locatedin different rooms in the structure than the A/V hub and the A/V displaydevice.

Then, the A/V hub provides, via an interface circuit in the A/V hub, thedisplay instructions (operation 1412) to the A/V display device fordisplay on the display.

Next, the A/V hub may optionally receive, via the interface circuit,user-interface activity information (operation 1414) from a portableelectronic device that specifies user selection of a content source. Forexample, the user-interface activity information may include activationof a virtual command icon displayed in the user interface on atouch-sensitive display in the portable electronic device.Alternatively, the user-interface activity information may includeactivation or changing a state or configuration of a physical button.Note that the A/V hub may optionally: generate user-interfaceinformation that specifies the user interface that includes one or morevirtual command icons, including the virtual command icon, which areassociated with the content sources; and provides the user-interfaceinformation to the portable electronic device for display on thetouch-sensitive display in the portable electronic device. However, inother embodiments the user-interface information is optionally generatedby the portable electronic device.

In response, the A/V hub may optionally perform one or more additionaloperations (operation 1416). In particular, the A/V hub may optionally:provide, via the interface circuit, a request for HDMI content to thecontent source; selectively provide, via the interface circuit, anactivation command to the content source based on device-stateinformation of the content source (such as when the content source isnot already turned on or activated); receive, via the interface circuit,the HDMI content from the content source; and provide, the HDMI contentand second display instructions to the A/V display device as frames withthe HDMI content are received from the content source, so that the HDMIcontent is displayed on the display in the A/V display device. In someembodiments, the A/V hub may optionally determine the second displayinstructions based on a format of the display. Note that providing theHDMI content and the second display instructions involves transcodingthe HDMI content based on the format of the display.

FIG. 15 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay device 114-1 and one or more content sources 116. In particular,processor 310 may determine display instructions 1510 specifyinginformation to be displayed on a display in an A/V display device 114-1.Then, processor 310 may optionally provide, via interface circuit 314,display instructions 1510 to A/V display device 114-1 for display on thedisplay, where the information to be displayed includes icons associatedwith content sources 116. Note that content sources 116 may be locatedat arbitrary or different locations in a structure, including locationsexternal to an environment of A/V hub 112 and A/V display device 114-1.

Moreover, processor 310 may optionally provide, via interface circuit314, user-interface information 1512 to portable electronic device 110.After receiving user-interface information 1512, portable electronicdevice 110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 1512 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide user-interface activityinformation 1514 that specifies user selection of one of content sources116 (such as user activation of a single command feature associated withthe one of content sources 116) to A/V hub 112. After receivinguser-interface activity information 1514, interface circuit 314 mayprovide user-interface activity information 1514 to processor 310. Inresponse, processor 310 may instruct interface circuit 314 to providerequest 1516 for HDMI content 1518 to the one of content sources 116. Inaddition, processor 310 may optionally determine display instructions1520 based on a format of the display in A/V display device 114-1.Alternatively, display instructions 1520 may be predetermined orpredefined.

After receiving request 1516, the one of content sources 116 may provideHDMI content 1518 to A/V hub 112. Furthermore, interface circuit 314 mayprovide HDMI content 1518 and display instructions 1520 to A/V displaydevice 114-1 as frames with HDMI content 1518 are received from the oneof content sources 116, so that HDMI content 1518 is displayed on thedisplay in A/V display device 114-1. Note that display instructions 1520may specify that HDMI content 1518 is to be displayed in a tiled windowon the display while additional HDMI content from another of contentsources 116 is displayed on the display. (Alternatively, in someembodiments interface circuit 314 provides HDMI content 1518 toprocessor 310, which instructs interface circuit 314 to provide HDMIcontent 1518 and display instructions 1520 to A/V display device 114-1as frames with HDMI content 1518 are received from the one of contentsources 116.)

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In particular, by allowing the user to see contentsources in a network throughout the environment that includes the A/Vhub regardless of their locations and/or the current state of thecontent sources (which may also be displayed), the display technique mayallow the user to select content sources at different locations in theenvironment and, thus, may provide location-independent operation. Inthe process, the display technique may reduce user frustration, and thusmay improve user satisfaction when using the portable electronic device,the A/V hub, the A/V display device and/or the one or more contentsources.

We now describe embodiments of the information displayed on the A/Vdisplay device (in conjunction with one or more virtual command icons inthe user interface on the portable electronic device) during method 1400(FIG. 14). FIG. 16 presents a drawing illustrating a display 1600 in anA/V display device (such as A/V display device 114-1 in FIG. 1). Inparticular, display 1600 shows a list of available content sources 1610at different locations in the environment.

When the user activates one of the virtual command icons associated withone of content sources 1610 (such as the DVR in the bedroom), the A/Vhub may provide display instructions and, after requesting and receivingHDMI content from the one of content sources 1610, may provide the HDMIcontent to A/V display device. As shown in FIG. 17, which presents adrawing illustrating a display 1700 in the A/V display device, the HDMIcontent may be presented in a tiled window 1710.

Embodiments of the display technique are further illustrated in FIG. 18,which presents a flow diagram illustrating a method 1800 for casting ofHDMI content. This method may be performed by an A/V hub, such as A/Vhub 112 (FIG. 1). During operation, the A/V hub (such as a controlcircuit or control logic, e.g., a processor executing a program moduleand/or or a circuit) receives, via an interface circuit in the A/V hub,user-interface activity information (operation 1810) from a portableelectronic device, where the user-interface activity informationspecifies a user instruction to cast the HDMI content associated with acontent source from a first A/V display device to a second A/V displaydevice (such as from a computer to a television). For example, theuser-interface activity information may include activation of one ormore virtual command icons (such as a single virtual command icon)displayed in the user interface on a touch-sensitive display in theportable electronic device. Alternatively, the user-interface activityinformation may include activation or changing a state or configurationof one or more physical buttons (such as a single physical button). Notethat the A/V hub may optionally: generate user-interface informationthat specifies the user interface that includes one or more virtualcommand icons, including the single virtual command icon, which areassociated with the first A/V display device and the second A/V displaydevice; and provides the user-interface information to the portableelectronic device for display on the touch-sensitive display in theportable electronic device. However, in other embodiments theuser-interface information is optionally generated by the portableelectronic device. Note that the first A/V display device and the secondA/V display device may be associated with different providers.

In response, the A/V hub accesses device-state information (operation1812) that specifies a current state of the second A/V display device.For example, as described further below with reference to FIGS. 21 and22, the device-state information may be determined using astate-detection circuit that is coupled to at least a pin in a port inthe A/V hub. Alternatively or additionally, the device-state informationmay be inferred based on device commands previously provided to thesecond A/V display device and a device-state diagram. In someembodiments, the device-state information includes a history of one ormore most-recent device commands previously provided to and received bythe second A/V display device (and, more generally, an entertainmentdevice) and/or a current state of the entertainment device based on thehistory of the one or more most-recent device commands and adevice-state diagram of the second A/V display device (and, moregenerally, the entertainment device). In some embodiments, thedevice-state information is stored in a computer-readable medium, suchas a memory.

Then, the A/V hub performs the casting (operation 1814) of the HDMIcontent from the first A/V display device to the second A/V displaydevice using a dynamic number of operations based on the device-stateinformation, so that the HDMI content is displayed on a display in thesecond A/V display device (which may be provided by a differentmanufacturer than the first A/V display device). Note that, from theuser perspective, the casting operation is from the first A/V displaydevice to the second A/V display device. However, in practice, thecasting operation involves the content source providing the HDMI contentto the second A/V display device. After the casting operation, the HDMIcontent may or may not still be displayed on the first A/V displaydevice.

In general, casting may include operations such as: turn on or power onthe A/V hub; turn on or power on the second A/V display device; set tocorrect input or content source; and/or enable casting. Once the userinstruction is received, the casting may occur in an automated manner.Note that when the device-state information indicates that the secondA/V display device is already powered on, a power-on operation may beexcluded from the dynamic number of operations. Moreover, when thedevice-state information indicates that the second A/V display device isalready set to receive the HDMI content from the content source, anoperation to set the second A/V display device to receive the HDMIcontent from the content source may be excluded from the dynamic numberof operations. In some embodiments, the casting may include receivingthe HDMI content from the content source, transcoding the HDMI contentbased on a format of a display in the second A/V display device, andproviding the transcoded HDMI content to the second A/V display device.

Note that the casting may be performed without further user action afteractivating the single virtual command icon or the single physicalbutton. (Thus, method 1800 may be used by the user to perform so-calledone-step or one-operation casting, even though it may involve the A/Vhub performing multiple commands or instructions.)

FIG. 19 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay devices 114-1 and A/V display devices 114-2. In particular,processor 310 may optionally provide, via interface circuit 314,user-interface information 1910 to portable electronic device 110. Afterreceiving user-interface information 1910, portable electronic device110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 1910 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide to A/V hub 112user-interface activity information 1912 (such as user activation of asingle virtual command icon) that specifies a user instruction to castthe HDMI content associated with a content source from A/V displaydevice 114-1 to A/V display device 114-2. After receiving user-interfaceactivity information 1912, interface circuit 314 may provideuser-interface activity information 1912 to processor 310.

In response, processor 310 may access device-state information 1914 ofA/V display device 114-2 in memory 1916 and/or may measure or determinedevice-state information 1914 using state-detection circuit 1918, adevice-state diagram and/or a history of previous device commandsprovided to and/or received by A/V display device 114-2. Based ondevice-state information 1914, processor 310 may dynamically adapt orchange a number of operations in a casting process 1920 based ondevice-state information 1914. Next, processor 310 may provide, viainterface circuit 314, display instructions 1922 to A/V display devices114-1 and display instructions 1924 to A/V display devices 114-2 toperform casting process 1920. Note that processor 310 may optionallydetermine display instructions 1924 based on a format of the display inA/V display devices 114-2. Alternatively, display instructions 1924 maybe predetermined or predefined.

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In particular, the display technique may allow the user,with minimum effort or operations, to cast content (such as HDMIcontent) from one A/V display device to another A/V display device. Inthe process, the display technique may reduce user frustration, and thusmay improve user satisfaction when using the portable electronic device,the A/V hub, and/or one or more A/V display devices.

Consequently, methods 200 (FIG. 2), 400 (FIG. 4), 1400 (FIG. 14) and/or1800 (FIG. 18) may reduce user errors or mistakes when using the userinterface, which may improve the user experience when using the portableelectronic device and/or the A/V hub.

In some embodiments of methods 200 (FIG. 2), 400 (FIG. 4), 1400 (FIG.14) and/or 1800 (FIG. 18), there may be additional or fewer operations.Moreover, the order of the operations may be changed, and/or two or moreoperations may be combined into a single operation. Furthermore, one ormore operations may be modified. For example, instead of involving theactivation of a single feature (such as a single virtual command icons),in some embodiments one or more of the operations involve the activationof one or more features (such as one or more virtual command icons).Alternatively or additionally, display instructions may be provided toan A/V display device differentially (such as when the displayinstructions change), regularly or periodically (such as in one of everyN packets or in a packet in each frame) or in each packet.

Note that in this display technique the A/V hub may display the HDMIcontent (and, more generally, video content) to an arbitrary A/V displaydevice (including an A/V display device that is located remotely fromthe A/V hub, such as in another room) without a need for a separateset-top box that is located proximate to the A/V display device.Instead, the A/V hub may perform all of the frame-by-frame transcodingof the video content that is needed for the A/V display device todisplay the video content before providing the video content to the A/Vdisplay device. Thus, in contrast with many existing cable and satellitesystems, the A/V hub may provide video content to multiple A/V displaydevices (such as N A/V display devices) without the use of N associatedset-top boxes. Consequently, the A/V hub may eliminate the need for aseparate set-top box in the same room as an A/V display device (althoughthere may be a local wireless receiver that is associated with the A/Vhub). This capability may be enabled by the knowledge of the devicestate information and the content selected by the users that isavailable to the A/V hub. In addition, this capability may eliminate theneed for a user to know where or how a particular A/V display device isconnected to a content source, such as cable television or a satellitedish.

We now describe embodiments of the information displayed on the A/Vdisplay device (in conjunction with one or more virtual command icons inthe user interface on the portable electronic device) during method 1800(FIG. 18). FIG. 20 presents a drawing illustrating a display 2000 in anA/V display device (such as A/V display device 114-1 in FIG. 1). Inparticular, after selecting a virtual command icon associated with anactive content source that is currently streaming content to the A/Vdisplay device and activating an associated casting virtual commandicon, display 2000 may show a list of A/V display devices 2010 at otherlocations. If the user selects another A/V display device in the list ofA/V display devices 2010 (such as A/V display device 114-2 in FIG. 1) byactivating a single virtual command icon in the user interface on theportable electronic device, at least some of the HDMI content displayedon the A/V display device (such as the HDMI content in optional tiledwindow 2012) may be cast to other A/V display device in the list of A/Vdisplay devices 2010. This casting may involve the A/V hub providing oneor more commands or instructions, including selectively turning on theother A/V display device (if it is not already turned on or activated),instructing the content source to provide a data stream with the HDMIcontent to the other A/V display device, etc. Thus, the one or morecommands or instructions may be based on device-state information aboutthe other A/V display device and the content source. Note that the HDMIcontent may or may not still be displayed in optional tiled window 2012.For example, when the casting operation is specified optional tiledwindow 2012 may no longer be displayed. Alternatively, optional tiledwindow 2012 may be displayed until the user activates a virtual commandicon to discontinue the displaying of the HDMI content.

As noted previously, the device-state information (such as whether anentertainment device is: electrically coupled to A/V hub 112 in FIG. 1,in a power-on state, in a power-off state, and/or another state, e.g., aplayback state, a pause state, a stop state, etc.) may be determinedusing hardware (such as a state-detection circuit) and/or software(which may be executed by a processor and, more generally, a controlcircuit). FIG. 21 presents a block diagram illustrating astate-detection circuit 2110 in A/V hub 112 (FIG. 1). In A/V hub 112(FIG. 1), input connector 2112 (which may be compatible with an HDMIstandard) may be electrically coupled to an entertainment device.State-detection circuit 2110 may be coupled to at least pin 2114 ininput connector 2112, so that, when the entertainment device iselectrically coupled to input connector 2112, state-detection circuit2110 establishes a ground loop between A/V hub 112 (FIG. 1) and theentertainment device. For example, pin 2114 may include a transitionminimized differential signaling (TMDS) data1 shield. (Alternatively,pin 2114 may include a TMDS data1 shield.) Moreover, state-detectioncircuit 2110 may include: an energy-dissipation component (such asresistor 2116) electrically coupled to a power-supply voltage and pin2114 (which may provide electrostatic-discharge protection); anenergy-storage component (such as capacitor 2118) electrically coupledto pin 2114 and ground; and a bi-directional voltage clamp (such asvaristor 2120 or a Verner diode), in parallel with capacitor 2118,electrically coupled to pin 2114 and ground. For example, resistor 2116may be 150 kΩ and capacitor 2118 may be 0.047 μF. In some embodiments,state-detection circuit 2110 includes a general-purpose input/output(GPIO) device 2126 coupled to pin 2114. The behavior (such as an inputpin, an output pin, enabled or disabled) of GPIO device 2126 may becontrolled using control signals or instructions from control logic2124.

FIG. 22 presents a flow diagram illustrating a method 2200 for detectingan entertainment device, which may be performed by an A/V hub (such asA/V hub 112 in FIG. 1) using state-detection circuit 2110 in FIG. 21.During operation, a control circuit (such as a processor and/or controllogic 2124, which may be included in or external to state-detectioncircuit 2110) in A/V hub 112 (FIG. 1), which is electrically coupled toinput connector 2112, detects whether there is electrical couplingbetween the entertainment device and input connector 2112 usingstate-detection circuit 2110 (FIG. 21). In particular, detecting whetherthere is electrical coupling between the entertainment device and inputconnector 2112 may involve: setting pin 2114 as an input (operation2210), where pin 2114 is then pulled to a power-supply voltage bycontrol logic 2124; measuring a voltage on pin 2114 (operation 2212)using control logic 2124; and detecting the electrical coupling betweenthe entertainment device and input connector 2112 when the voltage onpin 2114 is less than or equal to a predefined value (operation 2214),such as when the voltage is approximately ground, using control logic2124. Note that, when the electrical coupling between the entertainmentdevice and input connector 2112 is detected, control logic 2124 may: setpin 2114 as an output and electrically couple pin 2114 to ground(operation 2218), which may improve signal integrity; and measure asecond voltage (operation 2220) on hotplug-detect pin 2122 in inputconnector 2112. When the second voltage on hotplug-detect pin 2122 isless than or equal to the predefined value (operation 2222), controllogic 2124 may set pin 2114 as an input (operation 2224) and repeat themeasurement of the voltage on pin 2114 (operation 2212). Alternatively,when the voltage equals or exceeds a second predefined value (operation2226), such as when the voltage is approximately the power-supplyvoltage, control logic 2124 may repeat detecting whether there iselectrical coupling between the entertainment device and input connector2112. Furthermore, when the voltage is less than or equal to thepredefined value (operation 2214), control logic 2124 may identify acurrent state (operation 2216) of the entertainment device, such as: apower-off state, and a standby state. For example, control logic 2124may provide the set of first control commands, provide the set of secondcontrol commands, and/or may monitor (via one or more pins in inputconnector 2112) content activity, such as a data stream to and/or fromthe entertainment device. Thus, control logic 2124 may determine thatthe entertainment device is: in the power-off state when there is no adata stream; in the standby state when the data stream has a low datarate; and in the playback state when the data stream has a data rateassociated with A/V content and/or includes the A/V content. Note that,when the second voltage on hotplug-detect pin 2122 is less than or equalto the predefined value (operation 2222) and when the voltage is lessthan or equal to the predefined value (operation 2214), control logic2124 may repeat setting pin 2114 as the output and electrically couplingpin 2114 to ground (operation 2218).

When the electrical coupling between the entertainment device and inputconnector 2112 is detected, control logic 2124 may optionally attempt toidentify the entertainment device by providingconsumer-electronics-control commands (which may be compatible with anHDMI standard) to the entertainment device. Alternatively oradditionally (such as when the attempt is unsuccessful), control logic2124 may provide a set of first control commands associated withdifferent types of entertainment devices until, in response, contentactivity (such as packets or frames associated with a data stream ofcontent communicated to and/or from the entertainment device) isdetected by control logic 2124 via input connector 2112. For example,the set of first commands may include: a play command for the differenttypes of entertainment devices; and/or a trick-mode command (such asfast forward, reverse, fast reverse, or skip) for the different types ofentertainment devices. Moreover, when the content activity is detected,control logic 2124 may provide a set of second control commandsassociated with different providers of entertainment devices until achange in a state of the entertainment device is detected by controllogic 2124 via input connector 2112 and state-detection circuit 2110.The set of second control commands may include: power-on controlcommands for the different providers of entertainment devices; and/orpower-off control commands for the different providers of entertainmentdevices.

Alternatively or additionally, during operation control logic 2124 maydetect whether there is electrical coupling between the entertainmentdevice and input connector 2112 using state-detection circuit 2110 (FIG.21). When the electrical coupling between the entertainment device andinput connector 2112 is detected, control logic 2124 may: set pin 2114as an output and electrically couple pin 2114 to ground; and measure thesecond voltage on hotplug-detect pin 2122 in input connector 2112. Whenthe second voltage on hotplug-detect pin 2122 is less than or equal tothe predefined value, control logic 2124 may set pin 2114 as an inputand measure a voltage on pin 2114. Moreover, when the voltage equals orexceeds the second predefined value, control logic 2124 may repeatdetecting whether there is electrical coupling between the entertainmentdevice and input connector 2112. Furthermore, when the voltage is lessthan or equal to the predefined value, control logic 2124 may identifythe current state of the entertainment device. In some embodiments,control logic 2124: provides a control command to the entertainmentdevice; and identifies an additional state of the entertainment devicebased on content (such as A/V content) that is provided and/or receivedby the entertainment device in response to the control command. Forexample, the control command may include: a play command, and/or atrick-mode command (such as fast forward or fast skip, slow forward orslow skip, fast reverse, or slow reverse).

We now describe embodiments of an electronic device. FIG. 23 presents ablock diagram illustrating an electronic device 2300, such as portableelectronic device 110, A/V hub 112 or A/V display device 114-1 inFIG. 1. This electronic device includes processing subsystem 2310,memory subsystem 2312, networking subsystem 2314 and optional feedbacksubsystem 2334. Processing subsystem 2310 includes one or more devicesconfigured to perform computational operations. For example, processingsubsystem 2310 can include one or more microprocessors,application-specific integrated circuits (ASICs), microcontrollers,programmable-logic devices, and/or one or more digital signal processors(DSPs). One or more of these components in processing subsystem aresometimes referred to as a ‘control circuit.’

Memory subsystem 2312 includes one or more devices for storing dataand/or instructions for processing subsystem 2310 and networkingsubsystem 2314. For example, memory subsystem 2312 can include dynamicrandom access memory (DRAM), static random access memory (SRAM), and/orother types of memory. In some embodiments, instructions for processingsubsystem 2310 in memory subsystem 2312 include: one or more programmodules or sets of instructions (such as program module 2322 oroperating system 2324), which may be executed by processing subsystem2310. Note that the one or more program modules may constitute acomputer-program mechanism, such as a computer program or software.Moreover, instructions in the various modules in memory subsystem 2312may be implemented in: a high-level procedural language, anobject-oriented programming language, and/or in an assembly or machinelanguage. Furthermore, the programming language may be compiled orinterpreted, e.g., configurable or configured (which may be usedinterchangeably in this discussion), to be executed by processingsubsystem 2310.

In addition, memory subsystem 2312 can include mechanisms forcontrolling access to the memory. In some embodiments, memory subsystem2312 includes a memory hierarchy that comprises one or more cachescoupled to a memory in electronic device 2300. In some of theseembodiments, one or more of the caches is located in processingsubsystem 2310.

In some embodiments, memory subsystem 2312 is coupled to one or morehigh-capacity mass-storage devices (not shown). For example, memorysubsystem 2312 can be coupled to a magnetic or optical drive, asolid-state drive, or another type of mass-storage device. In theseembodiments, memory subsystem 2312 can be used by electronic device 2300as fast-access storage for often-used data, while the mass-storagedevice is used to store less frequently used data.

Networking subsystem 2314 includes one or more devices configured tocouple to and communicate on a wired and/or wireless network (i.e., toperform network operations), including: control logic 2316, interfacecircuits 2318 and associated antennas 2320. (While FIG. 23 includesantennas 2320, in some embodiments electronic device 2300 includes oneor more nodes, such as nodes 2308, e.g., pads, which can be coupled toantennas 2320. Thus, electronic device 2300 may or may not includeantennas 2320.) For example, networking subsystem 2314 can include aBluetooth networking system, a cellular networking system (e.g., a 3G/4Gnetwork such as UMTS, LTE, etc.), a universal serial bus (USB)networking system, a networking system based on the standards describedin IEEE 802.11 (e.g., a Wi-Fi networking system), an Ethernet networkingsystem, and/or another networking system. Note that the combination of agiven one of interface circuits 2318 and at least one of antennas 2320may constitute a radio. In some embodiments, networking subsystem 2314includes a wired interface, such as HDMI interface 2330 (which mayinclude a state-detection circuit).

Networking subsystem 2314 includes processors, controllers,radios/antennas, sockets/plugs, and/or other devices used for couplingto, communicating on, and handling data and events for each supportednetworking system. Note that mechanisms used for coupling to,communicating on, and handling data and events on the network for eachnetwork system are sometimes collectively referred to as a ‘networkinterface’ for the network system. Moreover, in some embodiments a‘network’ between the electronic devices does not yet exist. Therefore,electronic device 2300 may use the mechanisms in networking subsystem2314 for performing simple wireless communication between the electronicdevices, e.g., transmitting advertising or beacon frames and/or scanningfor advertising frames transmitted by other electronic devices asdescribed previously.

Within electronic device 2300, processing subsystem 2310, memorysubsystem 2312, networking subsystem 2314 and optional feedbacksubsystem 2334 are coupled together using bus 2328. Bus 2328 may includean electrical, optical, and/or electro-optical connection that thesubsystems can use to communicate commands and data among one another.Although only one bus 2328 is shown for clarity, different embodimentscan include a different number or configuration of electrical, optical,and/or electro-optical connections among the subsystems.

In some embodiments, electronic device 2300 includes a display subsystem2326 for displaying information on a display (such as the communicationwarning message), which may include a display driver, an I/O controllerand the display. Note that a wide variety of display types may be usedin display subsystem 2326, including: a two-dimensional display, athree-dimensional display (such as a holographic display or a volumetricdisplay), a head-mounted display, a retinal-image projector, a heads-updisplay, a cathode ray tube, a liquid-crystal display, a projectiondisplay, an electroluminescent display, a display based on electronicpaper, a thin-film transistor display, a high-performance addressingdisplay, an organic light-emitting diode display, a surface-conductionelectronic-emitter display, a laser display, a carbon-nanotube display,a quantum-dot display, an interferometric modulator display, amulti-touch touchscreen (which is sometimes referred to as atouch-sensitive display), and/or a display based on another type ofdisplay technology or physical phenomenon.

Furthermore, optional feedback subsystem 2334 may include one or moresensor-feedback mechanisms or devices, such as: a vibration mechanism ora vibration actuator (e.g., an eccentric-rotating-mass actuator or alinear-resonant actuator), a light, one or more speakers, etc., whichcan be used to provide feedback to a user of electronic device 2300(such as sensory feedback about the status of a user instruction tochange the state of one of the components in system 100 in FIG. 1).

Electronic device 2300 can be (or can be included in) any electronicdevice with at least one network interface. For example, electronicdevice 2300 can be (or can be included in): a desktop computer, a laptopcomputer, a subnotebook/netbook, a server, a tablet computer, asmartphone, a cellular telephone, a consumer-electronic device (such asa television, a set-top box, audio equipment, video equipment, etc.), aremote control, a portable computing device, an access point, a router,a switch, communication equipment, test equipment, and/or anotherelectronic device.

Although specific components are used to describe electronic device2300, in alternative embodiments, different components and/or subsystemsmay be present in electronic device 2300. For example, electronic device2300 may include one or more additional processing subsystems, memorysubsystems, networking subsystems, and/or display subsystems. Moreover,while one of antennas 2320 is shown coupled to a given one of interfacecircuits 2318, there may be multiple antennas coupled to the given oneof interface circuits 2318. For example, an instance of a 3×3 radio mayinclude three antennas. Additionally, one or more of the subsystems maynot be present in electronic device 2300. Furthermore, in someembodiments, electronic device 2300 may include one or more additionalsubsystems that are not shown in FIG. 23. Also, although separatesubsystems are shown in FIG. 23, in some embodiments, some or all of agiven subsystem or component can be integrated into one or more of theother subsystems or component(s) in electronic device 2300. For example,in some embodiments program module 2322 is included in operating system2324.

Moreover, the circuits and components in electronic device 2300 may beimplemented using any combination of analog and/or digital circuitry,including: bipolar, PMOS and/or NMOS gates or transistors. Furthermore,signals in these embodiments may include digital signals that haveapproximately discrete values and/or analog signals that have continuousvalues. Additionally, components and circuits may be single-ended ordifferential, and power supplies may be unipolar or bipolar.

An integrated circuit may implement some or all of the functionality ofnetworking subsystem 2314, such as one or more radios. Moreover, theintegrated circuit may include hardware and/or software mechanisms thatare used for transmitting wireless signals from electronic device 2300and receiving signals at electronic device 2300 from other electronicdevices. Aside from the mechanisms herein described, radios aregenerally known in the art and hence are not described in detail. Ingeneral, networking subsystem 2314 and/or the integrated circuit caninclude any number of radios.

In some embodiments, networking subsystem 2314 and/or the integratedcircuit include a configuration mechanism (such as one or more hardwareand/or software mechanisms) that configures the radios to transmitand/or receive on a given channel (e.g., a given carrier frequency). Forexample, in some embodiments, the configuration mechanism can be used toswitch the radio from monitoring and/or transmitting on a given channelto monitoring and/or transmitting on a different channel. (Note that‘monitoring’ as used herein comprises receiving signals from otherelectronic devices and possibly performing one or more processingoperations on the received signals, e.g., determining if the receivedsignal comprises an advertising frame, calculating a performance metric,performing spectral analysis, etc.) Furthermore, networking subsystem2314 may include at least one port (such as an HDMI port 2332) toreceive and/or provide the information in the data stream to A/V displaydevice 114-1 (FIG. 1) and/or one of the one or more content sources 116(FIG. 1).

While a communication protocol compatible with Wi-Fi was used as anillustrative example, the described embodiments may be used in a varietyof network interfaces. Furthermore, while some of the operations in thepreceding embodiments were implemented in hardware or software, ingeneral the operations in the preceding embodiments can be implementedin a wide variety of configurations and architectures. Therefore, someor all of the operations in the preceding embodiments may be performedin hardware, in software or both. For example, at least some of theoperations in the display technique may be implemented using programmodule 2322, operating system 2324 (such as drivers for interfacecircuits 2318) and/or in firmware in interface circuits 2318.Alternatively or additionally, at least some of the operations in thedisplay technique may be implemented in a physical layer, such ashardware in interface circuits 2318.

Moreover, while the preceding embodiments included a touch-sensitivedisplay in the portable electronic device that the user touches (e.g.,with a finger or digit, or a stylus), in other embodiments the userinterface is display on a display in the portable electronic device andthe user interacts with the user interface without making contact ortouching the surface of the display. For example, the user's interact(s)with the user interface may be determined using time-of-flightmeasurements, motion sensing (such as a Doppler measurement) or anothernon-contact measurement that allows the position, direction of motionand/or speed of the user's finger or digit (or a stylus) relative toposition(s) of one or more virtual command icons to be determined. Inthese embodiments, note that the user may activate a given virtualcommand icon by performing a gesture (such as ‘tapping’ their finger inthe air without making contact with the surface of the display). In someembodiments, the user navigates through the user interface and/oractivates/deactivates functions of one of the components in system 100(FIG. 1) using spoken commands or instructions (i.e., via voicerecognition) and/or based on where they are looking in the visualfeedback displayed on A/V display device 114-1 in FIG. 1 (e.g., bytracking the user's gaze or where the user is looking).

Furthermore, while A/V hub 112 (FIG. 1) was illustrated as a separatecomponent from A/V display device 114-1 (FIG. 1), in some embodimentsthe components are combined into a single component or a singleelectronic device.

While the preceding embodiments illustrated the display technique withaudio and video content (such as HDMI content), in other embodiments thedisplay technique is used in the context of an arbitrary type of data orinformation. For example, the display technique may be used withhome-automation data. In these embodiments, A/V hub 112 (FIG. 1) mayfacilitate communication among and control of a wide variety ofelectronic devices, including electronic devices in addition to or otherthan entertainment devices. Thus, A/V hub 112 (FIG. 1) and the displaytechnique may be used to facilitate or implement the so-called Internetof things.

Moreover, in the display technique different types of content may betreated similarly. For example, audio may be output and a correspondingimage or icon may be displayed in a tiled window. Furthermore, the otheroperations (such as casting) associated with the user interface and theinformation displayed on the A/V display device may have the same lookand feel and functionality as with HDMI content.

In the preceding description, we refer to ‘some embodiments.’ Note that‘some embodiments’ describes a subset of all of the possibleembodiments, but does not always specify the same subset of embodiments.

The foregoing description is intended to enable any person skilled inthe art to make and use the disclosure, and is provided in the contextof a particular application and its requirements. Moreover, theforegoing descriptions of embodiments of the present disclosure havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present disclosure tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art, and the generalprinciples defined herein may be applied to other embodiments andapplications without departing from the spirit and scope of the presentdisclosure. Additionally, the discussion of the preceding embodiments isnot intended to limit the present disclosure. Thus, the presentdisclosure is not intended to be limited to the embodiments shown, butis to be accorded the widest scope consistent with the principles andfeatures disclosed herein.

What is claimed is:
 1. An electronic device, comprising: an interfacecircuit configured to communicate with a display device and a portableelectronic device; and a control circuit, coupled to the interfacecircuit, configured to: determine one or more device states of one ormore content sources by monitoring, as a function of time, one or moredata streams associated with the one or more content sources, whereinthe one or more content sources are located at different locations in astructure, including locations external to an environment of theelectronic device and the display device, wherein the one or morecontent sources comprise at least one of: a DVD player, a Blue-rayplayer, a satellite receiver and cable television, and wherein themonitoring, as a function of time, involves calculating a number ofpackets or frames in the one or more data streams, is based on payloadsin the packets or frames, or both; dynamically adapt a user interfacebased on the determined one or more device states, wherein the userinterface includes one or more icons associated with the one or morecontent sources that indicate the one or more device states, wherein theone or more device states are selected from a set of multiple potentialdevice states including a power off state; determine displayinstructions specifying information corresponding to the user interfacefor display on a display in the display device; provide, via theinterface circuit, the display instructions intended for the displaydevice for display on the display; receive, via the interface circuit,user-interface activity information associated with the portableelectronic device, wherein the user-interface activity informationspecifies user selection of a content source in the one or more contentsources; selectively provide, via the interface circuit, an activationcommand for the content source based on the device state of the contentsource; provide, via the interface circuit, a request for HDMI contentfor the content source; receive, via the interface circuit, the HDMIcontent associated with the content source; and provide, via theinterface circuit, the HDMI content and second display instructionsintended for the display device as frames with the HDMI content arereceived via the interface circuit, wherein the HDMI content is fordisplay on the display in the display device.
 2. The electronic deviceof claim 1, wherein the electronic device includes a port that cancouple to the display device; wherein the port is compatible with ahigh-definition multimedia-interface (HDMI) standard; and wherein theinterface circuit is coupled to the port.
 3. The electronic device ofclaim 1, wherein the environment includes one or more rooms in thestructure; and wherein the different locations are located in differentrooms in the structure than the electronic device and the displaydevice.
 4. The electronic device of claim 1, wherein the electronicdevice includes one or more antennas communicatively coupled to theinterface circuit; and wherein the content sources are included in awireless network associated with the electronic device.
 5. Theelectronic device of claim 1, wherein the user-interface activityinformation includes information specifying user activation of a virtualcommand icon.
 6. The electronic device of claim 5, wherein the controlcircuit is configured to: generate user-interface information thatspecifies a second user interface that includes one or more virtualcommand icons, including the virtual command icon, which are associatedwith the content sources; and provide, via the interface circuit, theuser-interface information intended for the portable electronic devicefor display on a display in the portable electronic device.
 7. Theelectronic device of claim 1, wherein the user-interface activityinformation includes information specifying user activation of aphysical button.
 8. The electronic device of claim 1, wherein providingthe HDMI content and the second display instructions involvestranscoding the HDMI content based on a format of the display in thedisplay device.
 9. The electronic device of claim 1, wherein the controlcircuit is configured to determine the second display instructions basedon a format of the display in the display device.
 10. The electronicdevice of claim 1, wherein the control circuit is configured todetermine the display instructions based on a format of the display inthe display device.
 11. The electronic device of claim 1, wherein thecontrol circuit comprises: memory configured to store programinstructions; and a processor, coupled to the memory, configured toexecute the program instructions, wherein, when executed by theprocessor, the program instructions cause the electronic device toperform the operations of the electronic device.
 12. A non-transitorycomputer-readable storage medium for use in conjunction with anelectronic device, the computer-readable storage medium storing programinstructions that, when executed by the electronic device, cause theelectronic device to provide display instructions by performing one ormore operations comprising: determining one or more device states of oneor more content sources by monitoring, as a function of time, one ormore data streams associated with the one or more content sources,wherein the one or more content sources are located at differentlocations in a structure, including locations external to an environmentof the electronic device and the display device, wherein the one or morecontent sources comprise at least one of: a DVD player, a Blue-rayplayer, a satellite receiver and cable television, and wherein themonitoring, as a function of time, involves calculating a number ofpackets or frames in the one or more data streams, is based on payloadsin the packets or frames, or both; dynamically adapting a user interfacebased on the determined one or more device states, wherein the userinterface includes one or more icons associated with the one or morecontent sources that indicate the one or more device states, wherein theone or more device states are selected from a set of multiple potentialdevice states including a power off state; determining the displayinstructions specifying information corresponding to the user interfacefor display on a display in a display device; providing, via theinterface circuit in the electronic device, the display instructionsintended for the display device for display on the display; receiving,via the interface circuit, user-interface activity informationassociated with the portable electronic device, wherein theuser-interface activity information specifies user selection of acontent source in the one or more content sources; selectivelyproviding, via the interface circuit, an activation command for thecontent source based on the device state of the content source;providing, via the interface circuit, a request for HDMI content for thecontent source; receiving, via the interface circuit, the HDMI contentassociated with the content source; and providing, via the interfacecircuit, the HDMI content and second display instructions intended forthe display device as frames with the HDMI content are received via theinterface circuit, wherein the HDMI content is for display on thedisplay in the display device.
 13. The computer-readable storage mediumof claim 12, wherein the environment includes one or more rooms in thestructure; and wherein the different locations are located in differentrooms in the structure than the electronic device and the displaydevice.
 14. The computer-readable storage medium of claim 12, whereinthe content sources are included in a wireless network associated withthe electronic device.
 15. The computer-readable storage medium of claim12, wherein the user-interface activity information includes informationspecifying user activation of one of: a virtual command icon; and aphysical button.
 16. The computer-readable storage medium of claim 12,wherein the one or more operations comprise determining the seconddisplay instructions based on a format of the display in the displaydevice.
 17. The computer-readable storage medium of claim 12, whereinthe one or more operations comprise determining the display instructionsbased on a format of the display in the display device.
 18. A method forproviding display instructions, wherein the method comprises: by anelectronic device: determining one or more device states of one or morecontent sources by monitoring, as a function of time, one or more datastreams associated with the one or more content sources, wherein the oneor more content sources are located at different locations in astructure, including locations external to an environment of theelectronic device and the display device, wherein the one or morecontent sources comprise at least one of: a DVD player, a Blue-rayplayer, a satellite receiver and cable television, and wherein themonitoring, as a function of time, involves calculating a number ofpackets or frames in the one or more data streams, is based on payloadsin the packets or frames, or both; dynamically adapting a user interfacebased on the determined one or more device states, wherein the userinterface includes one or more icons associated with the one or morecontent sources that indicate the one or more device states, wherein theone or more device states are selected from a set of multiple potentialdevice states including a power off state; determining the displayinstructions specifying information corresponding to the user interfacefor display on a display in a display device; providing, via theinterface circuit in the electronic device, the display instructions tointended for the display device for display on the display; receiving,via the interface circuit, user-interface activity informationassociated with the portable electronic device, wherein theuser-interface activity information specifies user selection of acontent source in the one or more content sources; selectivelyproviding, via the interface circuit, an activation command for thecontent source based on the device state of the content source;providing, via the interface circuit, a request for HDMI content for thecontent source; receiving, via the interface circuit, the HDMI contentassociated with the content source; and providing, via the interfacecircuit, the HDMI content and second display instructions intended forthe display device as frames with the HDMI content are received via theinterface circuit, wherein the HDMI content is for display on thedisplay in the display device.
 19. The method of claim 18, wherein theenvironment includes one or more rooms in the structure; and wherein thedifferent locations are located in different rooms in the structure thanthe electronic device and the display device.
 20. The method of claim18, wherein providing the HDMI content and the second displayinstructions involves transcoding the HDMI content based on a format ofthe display in the display device.